Generally, in digital television broadcasting, a digital multiplexing scheme is employed that is specified in the MPEG-2 Systems standard defined in ISO/IEC 13818-1, “Information technology—Generic coding of moving pictures and associated audio information: Systems,” 2006, as one video and audio data multiplexing standard.
FIG. 1 is a diagram schematically illustrating the configuration of a video multiplexing apparatus that multiplexes video and audio data in the TS (Transport Stream) format specified by the MPEG-2 Systems. The video multiplexing apparatus 100 includes a video encoding unit 101, a video packet generating unit 102, an audio encoding unit 103, an audio packet generating unit 104, a control unit 105, a system packet generating unit 106, and a packet multiplexing unit 107.
The video encoding unit 101 encodes and compresses the input video data in accordance with a prescribed encoding method, for example, the AVC standard defined in ISO/IEC 14496-10, “Information technology—Coding of audio-visual objects—Part 10: Advanced Video Coding,” 2010. The video packet generating unit 102 packetizes the encoded and compressed video data into PES (Packetized Elementary Stream) packets. The PES packet format is such that a PES header carrying picture display time, etc., is appended for every one or for every predetermined number of pictures in the video data. The video data packetized into the PES packets is further packetized into TS packets each having a fixed length of 188 bytes. The TS packet format is such that a TS header is appended to a TS payload containing data into which a PES packet is split. The TS header contains payload identification information PID. The TS header further carries, for example, a program clock reference (PCR) value for reproducing the system time clock (STC) value of the video multiplexing apparatus 100. The PCR value must be strictly determined according to the insertion position (byte position) in the TS stream (TS packet stream). To this end, necessary information is exchanged between the video packet generating unit 102 and the packet multiplexing unit 107.
The audio encoding unit 103 encodes and compresses the input audio data in accordance with a prescribed encoding method, for example, the MPEG-2 AAC standard defined in ISO/IEC 13818-7, “Information technology—Generic coding of moving pictures and associated audio information—Part 7: Advanced Audio Coding (AAC),” 2006.
The audio packet generating unit 104 packetizes the encoded and compressed audio data into PES packets and TS packets. The audio PES packet format is such that a PES header is appended for every one audio frame (1024 samples).
The control unit 105 controls the operation of the video encoding apparatus 100. The control unit 105 has a 27-MHz STC which provides the reference clock for synchronization control. The control unit 105 determines the display time of each picture by taking the STC value at the vertical synchronization signal V-Sync timing of the input video data. It also determines the reproduction time of each audio frame by taking the STC value at the first sample timing of the audio frame in the input audio data.
The system packet generating unit 106 generates TS packets which store program specific information (PSI) as system information. The PSI includes a program association table (PAT) which carries a list of multiplexed programs, a program map table (PMT) which carries the PIDs of the video or audio data in each program, etc. The PID of the PAT is specified to be “0”.
The packet multiplexing unit 107 multiplexes together the video TS packets output from the video packet generating unit 102, the audio TS packets output from the audio packet generating unit 104, and the system TS packets output from the system packet generating unit 106. The packet multiplexing unit 107 thus produces a TS stream, which is output. The multiplexing of the packets is performed by conforming to the System Target Decoder (STD) specification defined in the MPEG-2 Systems standard. For example, the video TS packets are multiplexed so that each video TS packet will arrive at the STD at a time earlier than the display time specified by the PES header.
FIG. 2 is a diagram schematically illustrating the configuration of a multiplexed video decoding apparatus that decodes the video and audio data multiplexed in the TS format specified by the MPEG-2 Systems, and that displays the video data while reproducing the audio data. The multiplexed video decoding apparatus 200 includes a packet demultiplexing unit 201, a system packet processing unit 202, a control unit 203, a video packet processing unit 204, a video decoding unit 205, an audio packet processing unit 206, and an audio decoding unit 207.
The packet demultiplexing unit 201, working in conjunction with the system packet processing unit 202, identifies, based on the header information of each TS packet, the type of the data contained in the payload part of the TS packet. Then, the packet demultiplexing unit 201 passes the video PES packets to the video packet processing unit 204 and the audio PES packets to the audio packet processing unit 206, while passing the PSI information to the system packet processing unit 202. Further, the packet demultiplexing unit 201 passes the PCR value carried in the TS header to the control unit 203 together with its byte position information.
The system packet processing unit 202 analyzes the PSI information carried in the payload part of the system TS packet, acquires the PIDs of the video and audio data contained in the program to be decoded, and passes the PIDs to the packet demultiplexing unit 201.
The video packet processing unit 204 extracts the encoded video data contained in the payload of each video PES packet stored in the payload of the video TS packet received from the packet demultiplexing unit 201, and passes the encoded video data to the video decoding unit 205. Further, the video packet processing unit 204 passes the picture display time, etc., retrieved from the PES packet header to the control unit 203.
The audio packet processing unit 206 extracts the encoded audio data contained in the payload of each audio PES packet stored in the payload of the audio TS packet received from the packet demultiplexing unit 201, and passes the encoded audio data to the audio decoding unit 207. Further, the audio packet processing unit 206 passes the audio frame display time, etc., retrieved from the PES packet header to the control unit 203.
Like the control unit 105, the control unit 203 has a 27-MHz reference clock STC. The control unit 203 decodes the STC value based on the PCR value received from the packet demultiplexing unit 201, and performs processing for synchronization. When the picture display time and the audio frame display time received from the video packet processing unit 204 and the audio packet processing unit 206, respectively, are equal to the STC value, the control unit 203 instructs the video decoding unit 205 and the audio decoding unit 207, respectively, to perform decoding.
The video decoding unit 205 and the audio decoding unit 207 each decode the picture or audio frame in accordance with the decode instruction from the control unit 203, and output the decoded picture or decoded audio frame, respectively.